COMPLEX MANIFOLDS IN Q-CONVEX BOUNDARIES 
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O . Abstract. We consider a C°° boundary bQ C C" which is q- 

convex in the sense that its Levi-form has positive trace on every 
complex g-plane. We prove that b£l is tangent of infinite order to 
the complexification of each of its submanifolds which is complex 
tangential and of finite bracket type. This generalizes Diederich- 
Fornaess [2] from pseudoconvex to g-convex domains. We also 
\ readily prove that the rows of the Levi-form are ^-subelliptic mul- 

• tipliers for the 9-Neumann problem on q- forms (cf. Ho [4]). This 

fH , allows to run the Kohn algorithm of [5] in the chain of ideals of 

subelliptic multipliers for g-forms. If bfl is real analytic and the 
algorithm stucks on g-forms, then it produces a variety of holo- 
morphic dimension q, and in fact, by our result above, a complex 
g-manifold which is not only tangent but indeed contained in bQ. 
Altogether, the absence of complex g-manifolds in bQ produces a 
^ , subelliptic estimate on g-forms. 
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1. Complex (/-manifolds in the boundary and the Kohn 
algorithm on (/-forms 



Let Q be a smooth domain in C" defined by r = with dr ^ 0, 
and M a smooth CR submanifold of bfl of CR dimension q and CR 
codimension p. We assume that M is "complex tangential" to bfl in 
the sense that 



(1.1) TM C T v bQ. 



Condition (II. ip is familiar in the ambient of peak-interpolation sets. If 
M is minimal in the sense of Tumanov, it is endowed with a "wedge 
complexification" of dimension q+p, that is, a complex (q+p)- manifold 
W of wedge type with edge M (cf. [6]). When bfl is pseudoconvex, 
then W C bQ; this refines Bedford-Fornaess [T] which is in turn a 
development of Diederich-Fornaess [2]. In fact, according to |6J, W is 
made out of analytic discs attached to M. The pseudoconvexity of bQ 
brings the discs inside Cl and their complex tangency to bQ, which 
follows from ( 11. ip . brings them in bQ. For the last implication, we have 
just to apply Hopf Lemma to a plurisubharmonic Holder exhaustion 
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function of Q of type — (— r) v , for r\ close to 1, restricted to each disc. 
We weaken the hypothesis of pseudoconvexity and assume that bQ is 
g-convex, that is, for a choice of the Hermitian metric, the trace of 
the Levi form L^q = ddr\ T c b Q is positive on every complex g-plane of 
T c bQ, the complex tangent bundle to bQ. 

We strengthen the hypothesis of minimality and assume that M is 
of "finite bracket type" , that is, the subsequent brackets of C°° vector 
fields with values in T C M generate the whole tangent bundle TM. Note 
that when M is real analytic, finite type and minimality coincide. 

Theorem 1.1. Let bQ be q-convex and let M C bQ be complex tan- 
gential and of finite bracket type. Then W is tangent to bQ of infinite 
order along M. 

The proof follows in Section [2j 

The holomorphic dimension of a variety V C bQ at z Q is defined by 

(1.2) hoi dim 2 V = sup inf dim c (T c V n KevL m ), 

Uzo zeunv 

for U Zo ranging through the family of neighborhoods of z a . Remark that 
TV PI KerLbn is involutive; moving from z Q to a nearby point where 
the real and the CR ranks are constant, we may apply Frobenious 
Theorem and produce a foliation by smooth leaves of CR-dimension q. 
We select a leaf M, denote by C the Lie span of T C M, and observe 
that C C KerLbQ C T c bQ. By redefining z , if necessary, we may 
assume that C = TM; thus M is complex tangential and of finite type. 
Altogether, we have obtained 

Corollary 1.2. (i) Let bfl be q-convex and let V C bfl have holomor- 
phic dimension q at z a . Then, there is M C V of CR-dimension > q 
whose wedge complexification W is tangent of infinite order to bQ. 
(ii) If, moreover, bQ and V are real analytic, then W is contained in 
bQ and is a (complex) manifold not just a wedge manifold. 

Our purpose is now to run the Kohn algorithm in a g-convex domain 
and to show that, when it goes through, it produces a subelliptic esti- 
mate for g-forms. This requires a minor effort in adapting the proof by 
Kohn [5j in which the domain is pseudoconvex in the usual sense. 

We choose an orthonormal basis u>x, , oj n = dr of (1,0) forms, and 
the dual basis Lj of (1,0) vector fields. In this basis, we denote by (ry) 

the matrix of ddr and by u = ^ UjUj an antiholomorphic q form 

\j\=k 

with summation being taken over ordered multiindices \ J\ = q. The 
form is assumed to belong to the domain Dq* of d* that is, to satisfy 
uj\bQ = when n G J; we denote by C%°(Q PI U) q the space of g-forms 
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with support in a neighborhood U of a boundary point z Q G bfl with 
smooth coefficients up to bfl. We also denote by ||| ■ ||| e the tangential 
Sobolev norm (cf. [5]). 

Proposition 1.3. Let bQ be q-convex; then 
(1.3) 

\K\=k-l i j 

for any u G Dq* fl C™(Q n U) k , k > q. 

We express (II. 3ft by saying that each row of ddr is a |-subelliptic 
row-multiplier on fc-form. We use the notation Q(u,u) for the energy 
of the 5-Neumann problem, that is, the term in the right of (11.31) . 

Proof. We show that for any v G C™(U' H for U' DD U, and for 
any derivative D, we have 

(1.4) 

EE/ r ij u ii<Dv jK dV < Q{u,u) + E ll^i( v )ll 2 

+ E E / r ij v iKVj K dS. 

For D = Lk, (jl.4p follows from Schwartz inequality. For D = Lk, k < n, 
it follows from integration by parts, Schwartz inequality, and basic 
estimate for u. Finally, for D = L n , we write 

EE/ r ij u iKL n V jK dV = E E / r ij U iKV jK dS 
,^ |ftT|=jfc-lii=l,...,n^ n |X|=fc-l i,j<n^ bQ 

+ o((EliVII)iHD- 

Using again Schwartz inequality on 6fi for the positive 2-form fijUiKUjK 

\K\=k-lij=l,...,n 

over /c- vectors w, we get 

E | E r ij u iKVj K dV < ( £ E r ii U iKU jK ^ ' ( £ E ^U^i*) " , 
|l£"|=fc-l ij |if|=ft-lij=l,...,fi |Jf[=Jfe-ly=l,...,n 

and this yields (11.41) from (11.51) and the basic estimate. We use now 
(ll.4p for vjk = J2i r 'ij u iK- Reasoning as in [5] p. 97, we get 



L6 ) EE/ r ijUiKD(r kj u kK )dV <Q(u,u). 



|K|=fc-l ijfc 
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Using the microlocal factorization A 1 = Aa Aa for the tangential stan- 
dard elliptic psedo differential operator of order 1 (together with the 
fact that the different derivatives Z)'s represent the full A 1 ), we get 
(TOD from (Oil . 

□ 

We recall briefly the Kohn's algorithm. We define, in a neighborhood 
of z , the chain of ideals If C If C and of modules Mf C Mf C 
• •■M|, starting from 

(Mf = {dr, <);<),■}; , „ 

] = ^/r, det n _ q+ iMf where denotes the real radical, 
and, inductively, 

jM« = {MUdlLih 

By Proposition II. 3[ and by Garding inequality, Mf is made out of ~- 
subelliptic row multipliers (that is, (I1.3P holds) and If is an ideal of 
^-subelliptic multipliers over g-forms. By [5] Proposition 4.7, the full 
chain of M^'s (resp. J^'s) is made out of subelliptic row multipliers 
(resp. function multipliers). The proof of this point remains unchanged 
from pseudoconvex to (/-convex domains. 

We take our conclusions. If 1 G 1% for some h, then we have a subel- 
liptic estimate (for some e depending on the number h of steps and on 
the operation of radical) on g-forms and, in fact, on /c-forms for any 
k > q. If, instead, If l+1 = If t (and I\ does not capture 1), this reveals 
under the extra assumption bfl e C w , that V = V(I%), the zero-set of 
1%, has holomorphic dimension > q. By Corollary 11.21 this implies the 
existence of a complex g-manifold in bQ. Putting alltogether, we get 
the proof of 

Theorem 1.4. Assume that in a neighborhood of z , bfl is real analytic, 
q-convex, and contains no germ of holomorphic manifold of dimension 
> q. Then a subelliptic estimate in degree k > q for the d -Neumann 
problem holds in a neighborhood U of z Q , that is, for some e we have 

\\u\\ 2 € < Q(u,u) for any ue D B , n C?{pL fl U) k . 
Example 1.5. In C 3 , consider the domain fl defined by 

x 3 > -\zx\ 2 \z 2 \ 2 + (^ki| 4 + ^l^l 4 ). 

Here bfl is real analytic, there are no complex 2-manifolds at but 
just the complex curve defined by z\ = z 2 . Also, if we compute the 
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Levi form of bfl in the metric in which 7r~ 1 (l, 0, 0) and 7r^ 1 (0, 1, 0) (for 
n z : T z bQ — > C 2 x R being the projection along the X3-axis) is an 
orthonormal system for T^bQ, we have 



It follows 



I z 2 1 2 + I z l 1 2 ~~ z l z 2 



~ z l z 2 ~ \ z l | 2 + 3 1 2^2 ' 2 



trace L b Q = 2\z 2 \ 2 > 0. 

Thus we have a subelliptic estimate in degree 2 according to Theo- 
rem 11.41 Note that this example could not be explained neither by 
usual pseudoconvexity nor by strong 2-pseudoconvexity. In fact 

detL 6Q = (\zi\ 2 - N 2 )(3|z 2 | 2 - \ Zl \ 2 ) - N 2 N 2 

= -hX - 3\z 2 \ A + 3\ Zl \ 2 \z 2 \ 2 



< 0. 



Thus, 



• bfl is not pseudoconvex (because det L b Q < implies that there 
are eigenvalues of opposite sign), 

• bQ does not satisfy Z(2) (in the sense of [3]) because there are 
no positive eigenvalues at 0. 

2. Proof of Theorem 11.11 

We adapt the proof of [2] Proposition 3 to the new situation in which 
bfl is no more pseudoconvex but just g-convex. We move to a nearby 
point that we still denote by z at which the "multitype" in the sense 
of (i)-(v) below is minimal (in the lessicographic order). We observe 
that the wedge complexification W can be (non-uniquely) continued 
to a smooth manifold without boundary W of real dimension 2(q + p). 
Since W is holomorphic, then W is "approximatly holomorphic" at 
M. By a linear unitary coordinate change we can assume that z Q = 0, 
T Zo M = OxRPx{0} andT Zo W = CxC p x{0} andT 2o 6fi = C n_1 xiR. 
We observe that the projection 7r along the z n -axis is transversal to W; 
thus n(W) and 7r _1 7r(VF) are real manifolds of dimension 2(q +p) and 
2(q + p + 1) respectively. We use the notation t := n — {p + q + 1). 
We suppose that 7r~ 1 7r(W / ) is defined by real equations fij(z') =0, j — 
1, 2t such that, putting fj =: fij + i^t+j, j < t, we have dfj = O^, 
and W is graphed over 7r(W) by z n = h + ig with d(h + ig) = O^; 
here denotes a zero of infinite order at M. Clearly M is defined by 
x n — h = 0, y n — g — 0, p = 0, /i = (where by p and fi we denote 
the full set of the pjS and /x^'s). We consider the Hermitian metric on 
C n in which Q is g-convex and the induced Euclidian metric on M 2n . 
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In this metric, we choose an orthonormal basis {-Xo,i}f=i of T C M and 
a completion to a full basis of TM 

s 

(2.1) {X 0> i}£i, {X 1>l }t 1 , {X s ,}ti with p = 2q and J> = p. 

i=i 

We may assume that 

(i) any j-iterated bracket of the X /s is in the span of the X h /s 
for h < j, 

(ii) Xj ti = [X 0>v , Xj_i tfi ] modulo Span{X ? ' j}j'< J _ 1 for suitable 
X 0>v e Span{X 0ii } and Xj_ 1>fl G Span{X /l)i }^< j _i when j > I. 

This is an immediate consequence of Jacobi identity. We put C° = 
T C M, write, inductively, CJ = Span{£ J_1 , [X , v , Xj-i^]}^ and de- 
compose 

TM = £°©^©...©-^ T . 
We can assume that our linear unitary tranformation gives L ' 



a- 1 

{0} x W j x {0}. Also, we can choose our basis so that, in addition to 
(i)-(ii) we also have 

(iii) each group {Xj^i 

=i,...,Pj is 

orthogonal one to another for dif- 
ferent j. 

(iv) in a basis z ,i, —Zo, q , Zi,i, -Zi, pi , ■■■ of C q+P we have X 0:i \ Zo = d Xi) 
X ,i+ q \z o =d Vi , i<q, and X jti \ Zo = d Xji for j>l, 

(v) M is the intersection of W with the set defined by pj^ = 0, j > 
1, where the Pj/s are functions on tt(W) with Span{Re9pj j} = 
Span { Re cfa/j ; j} and with (dphj, Lj ti ) = for any h > j + 1. 

Note that, in particular, (v) implies that ddph(Lji,Lji = for any 
j- f <h-2. 

We identify the Xj ti e TM to the real or imaginary parts of vector 
fields Lj i e C(TM + \jTM) n T 1,0 C" defined by 
(2-2) ' 

{To,i := A" 0i i+zXo ig+ i}j =1 , {Li t i := Xi^+iJXi^}^, {L Sj j := Xg^+iJXs^}^. 

Since C(TM + JTM) n T 1 '°C™ C r 1,0 6Q| M , we extend the L = L jyi 
from M to the whole bQ as sections of T 1,0 bQ keeping unchanged their 
notation. We can also arrange that the Lj^ are extended from M to W 
so that (dpj, L) = O^, j = 1, ...,2t. For that, we extend them with the 
request (dfj,L) = 0, j — l,...,t; since (dfj,L) = O^, the conclusion 
follows remembering that the //'s are the real and imaginary parts of 
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the /'s. By (iii) above, and by the fact that C° is invariant under 
J, we have that the Zy$, j > 1, are orthogonal to C£°; this stays 
true also outside M for the extended vector fields. Moreover, possibly 
after renormalization, the L 0ii can be chosen so that they form an 
orthonormal system. 

Recall that for the equation z n = h + ig of W, we have supposed 
d(h + ig) = O^j and thus, in particular, ddh = Of 4 . Thus, if bfl is 
graphed by x n = h + a (which serves as a definition of cr), we have 
Lbn = dda\ T x,o b Q + O^. We also denote by r := x n — (h + a) a definig 
function for bQ. Note that cr = on M; we want to prove that 

a = 0(p°°) when y n — g = and fi = 0, 

and hence W is tangent of infinite order to bQ along M. We expand 

(2.3) o = J2 a iP* + 0(p k+1 ) +£ + £ u 

\i\=k 

where / is a multi bi-index in the (j, i)'s and where £ = 0(y n — g) and 
£\ = 0([i). We observe that 

(2.4) dd£(L,L) = 0((dz n ,L}(dp,L}) + \(dz n , L)\ 2 + 0(y n - g). 

In particular, recalling that (dr,Lj ti ) = and (dp,L 0ji ) = O(p), we 

have for y n — g = 

(2.5) 

dd£(L j;U l jV ) = cV- 1 ), a<9£ (Lj-i, z ,i') = o(p fc ), aa^(L 0ji , Zo,<0 = 0(p fc+1 ). 

As for dd£\, recalling also {dp, L) = Ofj, we have 
(2.6) 

dd£i(L, L) ~ ddp(L, L) + | (dp:, L) \(\(dp, L)\ + \ (dz n , L)\) + O(p) 
= 0^ + 0( P ™) + 0(p). 

For this reason, when evaluating dda on L as above, we can assume 
without loss of generality that £\ = in (12.31) . We call k the first integer 
for which there is in (12. 3p a non-trivial occurence aj for |/| = k; we wish 
to show that k cannot exist finite. First, the inclusion TW\m C T c bQ\M 
implies k > 2. We first show that k cannot be odd. In fact, by a choice 
of L = X + iJX, X G C? i j > 1 such that dda(L, L) is obtained by 
differentiating two factors once, we get 

(2.7) dda(L,L)= arp 1 ' +0(p k - 1 ) + dd£(L,L), 

\I'\=k-2 

with ap 7^ for at least one I'. By the first of (12.51) . the last term in 
(12.71) can be neglected. Thus the form in the right of (12. 7p . having odd 
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order, it changes sign. On the other hand 

(2.8) dda{L^L ^) = O(p k - 1 ). 

Define a g-plane by Q q := Span{L, L i }i (for any choice of q—1 between 
the indices i); we can conclude that traceQ dda changes sign, which 
violates the (/-convexity of 60. Thus k cannot be odd. 

We show that k cannot be even, either. We first remove any possible 
term with a factor of p\^ in the homogeneous expression of degree k of 
a, that is, Yl a {i,%)l'P 1 ■ We have 

(2.9) dda\ cc0 = (^«(MK'^PMlc£o)p / '+0(p fc ) + «| C£ o. 

\I'\=k-l i 

By the third of (12.51) . the last term in (12. 9p can be neglected. If, for some 
\I' \ — k— 1, we have tracec£o(^V a^^i' ddpi^) ^ 0, then tracer (dda) 
changes sign since the p 1 ' vary independently. 
Otherwise, assume 

(2.10) trace C£ o ( ^ a^pddpiA = for any V . 

i 

Recall that the commutators of the L' 4 s span a space of dimension pi, 
by Cartan formula, this is equivalent as to saying that the Levi matrices 
<9<9pi i j| C £o, i = l,...,p 1 are independent. Thus, from £\ a^i^pi^ ^ 
for some I' Q , we get for some vector of CX°, say Lo,i> 

(2.11) ^ a (i,i)i' o d 9pi,i(L 0il , Z ,i) ^ 0. 

i 

Define L t = ( 1 ~ t )- L °. 1+f Ll -* (any i) where c t is a factor which normalizes 
\U t \ = 1. We deform CC° to 

Qq = Span{L t , Lo,2, • 
Combination of ( ED} and ( l2"TTTj) yields 

traceQ,(^0( 1)i )//99pi, < ) = tej* for cj/ ^ 0. 

i 

Then, using ( 12. 4p . we have for the trace of the full a = Y^\i\>k ai P 
(2.12) 

tracer = i Cj ,/»+ ^ c r p I '+t i O(^- 2 )+^0(j/ t - 1 )-^(j/ t )+dd£\ Qq -, 

IJ'l=fc— 1 

observing that by (12.51) we have dd8\ Qq = t 4 0(p fc_1 ) + t 2 0(p k ) + 
0(p fc+1 ), we see that this term can be neglected. By taking restric- 
tion to a suitable region of the plane R r x 1 of (pjj,t), all terms in 
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the right of ( 12. 12ft are negligeable comparing to the first: thus, again, 
traceg (ddcr) changes sign. 

At last, we have to consider the case when Y2\i\=k a iP I contains fac- 
tors pj Bj i which start from j > 1. For fixed h, each group of matrices 
ddph,i, i = l,...,Ph, are independent. Thus, for a pair of vectors, say 
Lqi G C£° and % -i,i G C£ J '° -1 , and for some \I' \ = k — 1, we 
have (J2i a (jo,i)i' ddPjo,i)(Lo,i,Lj o ^i t i) ^ 0. But then, under the choice 

L := - — Lo ' 1 ^ L: '°~ 1 ' 1 ; t « 1, (for a normalization factor c t ) we have 

(2.13) (J2 a {jo>{]I ,dd Pj J{L, L) = Cl , + 0. 

i 

We then complete L by q — 1 vectors in C£° to an orthonormal basis 
of a g-space Q q thus obtaining 

(2.14) trace Q9 (daa) = tc r /° + t ^ C/ //' + t 2 0(p k - 1 ) + 0(p fc ), 

|/'l=fc-i 

where tO(p k ~ 1 ) comes from differentiation once with respect to L dif- 
ferent terms pj ot i in (k + l)-powers and where we have controlled the 
term dd£ by t 2 0(p k ~ r ) + 0(p k ) + 0(p k+l ). Again, we can make ne- 
gleageable in the right of ( 12. 14ft the terms which follow the first and 
conclude that the trace changes sign, a contradiction. 

In conclusion, k cannot exist neither odd nor even and therefore a 
vanishes of infinite order along M . 
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